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Abstract 

We investigate the Tits buildings of the paramodular groups with or without canonical 
level structure, respectively. These give important combinatorical information about the 
boundary of the toroidal compactification of the moduli spaces of non-principally polarised 
Abelian varieties. 

We give a full classification of the isotropic lines for all of these groups. Furthermore, 
for square-free, coprime polarisations without level structure we show that there is only 
one top-dimensional isotropic subspace. 

In a sequel to this paper we will use this information to establish a general type result 
for the moduli space of non-principally polarised Abelian varieties with full level structure. 

1 Introduction 

We fix (ei,...,e g ) with e,|e,+i for all i = 1, ... ,g — 1. Let A := diag(ei , . . . ,e g ), A := 
(-A o)- Let £ := I? 8 C C 8 and let £ v be the lattice dual to £ with respect to the bilinear 
form (x,y) :=xA^, namely 

£ v :={ye£<g>Q|Vxe£ : (x,y) e Z}. 

Recall that the paramodular group, respectively the paramodular group with a canonical 
level structure can be defined as follows: 

f P oi := {M 6 SL(2g, Z) \MA'M = A} and 
fpol := {M e r pol |M| £ v /£ = id | £ v /£ }. 

Their action on the Siegel upper half space & g is given by 

(cd) (At + BA)(Ct + DA)-'A. 

The quotient spaces i? po i := 6g/r po i and A™^ := &g/T 1 ^ are the moduli spaces of Abelian 
varieties with fixed polarisation of the given type without or with canonical level structure, 
respectively. It is well known 1 that these groups are conjugate to subgroups of Sp(2g,Q), 
namely r po i := R~ l T pol R and := R-^T^R where R := ( 1 A ). The action of these 
groups on & g is the one induced from Sp(2g,Q), namely 

:th (Az + B){Cz + D)-\ 

and the respective quotient spaces are isomorphic to A po i and respectively. All of 
these groups also act on Q 2g by matrix multiplication from the right. 

A subspace V C Q 2g is called isotropic if for all u,v € V we have (u,v) — 0. The Tits 
building describes the configurations of the conjugacy classes of these isotropic spaces with 
respect to the action of the different groups defined above. It provides useful information 

'see eg. [HKW, p. 11] 
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about the combinatorical structure of the boundary components of toroidal compactifica- 
tions. 2 Instead of considering the whole building we focus on the one- and g-dimensional 
spaces only. 

The main results of this paper are the classification of isotropic lines in Corollary 4.2 
and Corollary 4.6, and Theorem 5.3, which says that under some conditions there is only 
one top-dimensional isotropic space. 



2 Divisors of vectors 

Let us begin the analysis of the Tits building by the one-dimensional isotropic subspaces 
of Q 2g . Given a polarisation type (e\ , . . -,e g ), we may chose e\ = 1 without changing the 
group fp i. 

Notation 2.1. 

Let di := e,+i/e, for i = 1,. . . ,g — 1 and define 



d t j := 



i f=Yi i=i d n for i < j 
1 for i > j 



Then all di are positive integers and the polarisation type is given by [\,d\,d\-2,- ■ ■ ,d\:g-\)- 
Let T(s) be a function depending on some integer variable s. Then we define 

gcd (T(s)Y s=i := gcd (7X0, • • • ,T(jj) and T(si l«) : = 8 cd (T(si),T(s 2 )) . 



Definition 2.2: Special polarisation types. 

We call a polarisation type (1, d\, . . . ,d\ :g -i) square-free if all di are square-free. If a 
polarisation type satisfies gcd (<f ,,£/,) = 1 for all ;' ^ j we call it a coprime polarisation 
type. 

First of all, we define the divisors A'(v) of a vector v e I? g for i = 1, . . . ,g — 1. To 
keep the notation easier, we shall drop the vector v where possible and write D, := D;(v). 

Definition 2.3: Divisors. 

Define the divisors D; := D;(v) of a primitive vector v e I? g recursively: 

A:=gcd(^gcd(^)^) e N >0 . 
Here, is defined as a product, analogously to di-j. 
Definition 2.4: Ideal of lattice and vector. 

For a vector v G Z 2g let (v, £) := { (v, I) | Z e £} which is an ideal in Z, namely 

(v,£) = {vi,diV2,du2V3,---,dl:g-lVg,Vg + i,diV g+ 2,...,di:g-iV2g) C Z. 



Lemma 2.5. 

For 1 <k<i<g and m such that D — j 6 Z, the following equivalence holds: 
Proof. 

The implication "=>" is trivial; the other direction can be proved by induction, substituting 

m' = dk-im. □ 

2 see [AMRT] 
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Lemma 2.6. 

The ideal (Z) ( ) can also be given by 
Proof. 

Let A, := (di, ^\ ) • We know from the definitions that 

^(,, gcd( ^)«J 

Since now all terms in (2) are multiples of terms in the definition of D,-, we obviously have 
A; C (A')' F° r m e other inclusion we apply Lemma 2.5 to every element in (2) to obtain 
that all terms in the definition of D, are contained in A,. □ 

Corollary 2.7: Invariance. 

The divisors D, of a vector v are invariant under the action of r po i on I? g . 
Proof. 

Consider the invariance of (v, £) under the action of M G r po i: 

(vM,£) = {vMA'l\l e Z 2 «} = {vA('M)- lr /|/ e Z 2 «} = {vA f /|/ e Z 2 «} = (v,£). 

This holds because ('M) _1 is an integer matrix due to det(M) = 1, and MA = A('M)~ l by 
the definition of r po i. The invariance of D, follows from Lemma 2.6. □ 

Remark 2.8. 

Let us point out that the divisors D, are not independent and therefore not every possible 
combination of divisors of the <f, given by the polarisation type can actually occur. E. g. 
take g = 3 and d\ = 4,<f 2 = 6 so that we have a polarisation type (1,4,24). Now, there is 
no vector with the divisors D\ = D2 = 2 because that would mean that 

(3) Di=gcd(4,vi,v 4 )=2 and 

(4) D 2 =gcd(6,^,^,V2,v 5 )=2, 

where equation (4) clearly shows that 4 divides both v\ and V4, which is a contradiction to 
(3). The additional restriction on the divisors D, is the following: 

Theorem 2.9: Restrictions on D ( . 

For 1 < i < j < g — 1 we have 

(5) gcd(f,D,) = l. 

Moreover, any ordered set of positive integers {D,} := {D\ , . . . ,D g -\ } satisfying D, |<i ( - and 
condition (5) does occur as set of divisors of a vector v e Z 2 *. 

Proof. 

Necessity: 

Take i < j and assume n := gcd(-^-,D 7 ) 7^ 1. We claim that any power of n divides di in 
contradiction to di ^ 0. The proof is by induction. 
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Define the index set / := {1, . . . ,i,g + 1, . . . ,g + i} and let 

v[ 0) : = v k for k e I, df ] :=d h D, (0) : = D, and 

W V k ,{r) di (r) Di 

v y >:=—, d ■':=— , D) := — forr>l. 

We want to show that all values we just defined are integers. For the generation r = this 
is obvious. 

Assume that all values of the generation r — 1 are integers. By definition of n we know 
that n divides ^ and hence 

Di n^D [ r l) D [ r l) 



Furthermore, n divides Dj by definition. After cancelling n r 1 in the first terms of the 
definition of Dj we obtain that n divides 

D > =**(*»**( V U )[ Mir^i +, 

This shows that in particular 

(r-l) 
V k\e+k 

(6) n divides -r~T\ for = 1 , . . . , i 

Dtj-tf]" l) D i+U j-i 

(r—l) (r) 

and hence also n\v\ which implies Vj eZ for all k 6 /. Furthermore, since obviously 

(r—l) 

Dk:i-\\Dk;i-\D\ 'Di + \-j-\ statement (6) also implies that n divides 

So, we have shown that all values in the rth generation are integers. The contradiction 

follows as mentioned above. 

Sufficiency: 

Choose integers D, satisfying the conditions stated in the lemma. Consider the vector 

v = (D 1;H ,D2 ;r i r ..,D H ,l 1 I ...,0)eZ 2 «. 

It is easy to calculate that the divisors D,-(v) are exactly the chosen D ( . □ 

This lemma has an interesting consequence: 
Corollary 2.10: Characterising property of D\ :g -\. 

For a given polarisation type (l,c/i, . . . ,di :g -i), the value Di :g _i(v) determines all the val- 
ues D,-(v) uniquely. 

Proof. 

Let di,.. -,d g -\ and Di- g -\ be given. Then Theorem 2.9 leads to the following: 

gcd(^,D 2 ) = -..=gcd(^,D,_ 1 ) = l =>» gcd(^,Z) 2w _ 1 ) = l 

=> gcd(di,Di :? _i) =Di 

so that we can determine D\ from and Di :g -\. Divide D\- g _i by Di to obtain D2 g -\ and 
apply the same lemma. By iterating this method all values D, are obtained. □ 
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3 Properties of symplectic matrices 



We now want to investigate divisibility properties of the matrix entries ofMeT for the 
different groups F we defined. 

Definition 3.1: Triangular polarisation matrices. 

Define the sets of matrices 

D(A) := {( Sy ) G Z* x «| j < i clju ,|.v } and 
SB(A) := D(A) n SL(g, Z) = {S G B(A)| det(5) = 1 } . 

Lemma 3.2. 

The set D(A) with the normal matrix operations is a ring with unity. Its subset SB (A) is a 
multiplicative group. 

Proof. 

It is a straightforward computation to check that B(A) is indeed a ring with unity. 

Since SB (A) C SL(g, Z) per definition, it is obvious that for any S G SB (A) the inverse 
T := S~ l exists, is an integer matrix and has determinant 1. It remains to show that T = 
(tij) G B(A). By Cramer's rule we know r, 7 = r^S^I = |5^| where is the minor 
of S constructed by removing the jth row and ;'th column. 

For j > i there is no additional condition. Now let j < i and fix n G {j, . . . , i — 1}. We 
have to show that d„ divides det(S^^). This is the statement of Lemma 6.1 where we let 
d = d„ and k = n. Using this for all j < n < i— 1 we obtain dj-j-i |fy which completes the 
proof that T G SB(A). □ 

Lemma 3.3. 

We have the following congruence conditions: 

f pol G B(A) 2x2 

where B(A) 2x2 := { (£ g) \A,B,C,D G B(A)}. 
Proof. 

Let M = (rriij) G f po i and chose k £ {1,. . . ,g — 1}. Denote the index set 

I k :={k+l,...,g,g + k+l,...,2g}- 

Now chose any i G 4 and let v : = e, G I? g be the ith unit vector. The invariance under the 
action of f po i and some easy computation shows that 

d k = D k (v) = D k {vM) = gcd (d k ,^J_,^i±l) . 

v I'jk i I'll i • 

This reasoning for all valid combinations of values leads exactly to the divisibility condition 
forM G B(A) 2x2 . " □ 

Lemma 3.4. 

For the conjugate group we have 

o ,„ ^ ( B(A) B(A)A \ 

rpoi = s P (2,,Q)n^ A _ 1 ^ A) A _4 ( J A)A j. 

Proof. 

This follows from Lemma 3.3 by conjugating with R. □ 
For the groups with canonical level structure we obtain additional conditions: 
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Lemma 3.5. 

f% = {m e r pol |M e (Q) i 2g ) ®z + i} 

where I) := (l,Ji,. . . ,<fi :s _i) and l2# := (1, . . . , 1) € Z 2g . The tensor denotes that each 
matrix entry of the rank 1 matrix in brackets may be multiplied by an integer ztj. 
Proof. 

Denote the obvious basis of £ C C 8 by {e\ , . . . ,e2 g }- Then a basis of the dual lattice £ v 
can be given by {-^— -e ( | g+ ;};=i,..., g . By definition, a matrix M £ f po i is in f p e J[ if and only 
if it satisfies M £ v j Z = id £ v y £ . This is satisfied if and only if for all i = 1 , . . . ,g we have 

1 1 , 

ei\g+iM = £ e Ag+i <^=> - e Ag+i {M-t) £ Z 8 



This means that divides every entry in the ;th and g + ith row of the matrix M — 1 

which is exactly the condition we wanted to prove. □ 



Lemma 3.6. 



r p e v i = {m e r pol |M e (i g ,v)) ® z + 1} 



where again := (l,d\,. . . ,di :g -i) and l g := (1, . . . , 1) 6 Z 8 . 
Proof. 

This follows directly from Lemma 3.5 by conjugating with R. □ 

One important result from this lemma is the following observation: Although r po i 
may have rational non-integer entries, this is no longer possible for its subgroup Tp 6 ^: 

Corollary 3.7. 

r^csp^z). 



Proof. 

With Lemma 3.4 we know rj^ C r po ] C Sp(2g,Q), and since the condition given in 
Lemma 3.6 implies that all matrix entries must be integers the claim follows immediately. 
□ 



4 Orbits of isotropic lines 

In this section we construct two sets of vectors that are in one-to-one correspondence to the 
orbits of the group actions of f po i and f p e J[, respectively. 

4.1 Orbits of isotropic lines under f jj^ 

Lemma 4.1: Orbits of isotropic lines under f p e J,. 

(i) . Under the action of f p e J[, every vector v £ I? 8 can be transformed into 

v= (Di: g -i(v), *,..., *,0, *,...,*) 
where the given is at the g + 1st place. 

(ii) . Two vectors v, w £ I? 8 are conjugate under f p e J[ if and only if 

Di-.g-i(v) =D\ :g -i(w) and Mi = 1, . . . ,2g : v ( - = w, modDi ;? _i. 
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Proof. 
Part (i): 

Since v is primitive, not all entries v, are zero. Hence we can assume (if necessary after a 
suitable transformation with a matrix in f jjjj) that v\ ^ 0. By definition of D, we know that 

^rr^- G Z for j <k < g and so it makes sense to say 

u j:k 



where the second gcd is equal to 1, again by definition of D ( . Hence, we also have 

di 



(7) 

Now define 

(8) 



/-= / V l|g+1 ^1 y 2|g+2 d\ d 2 : g -lV g \ 2 g \ 



Using (7) for i = 1, . . . ,g — 1, we may drop the factors ^ successively to obtain 

T _f v l\g+1 V 2\g+2 v \ 



and since gcd(vi , . . . , V2#) = 1 we have / = (1 ). With Lemma 6.2 we can now find A,,- such 
that 

( „ , 7: + L X i n + L V' = ( 1 ) or equivalently 

(vi,v g+ i+ {hdhi-iVi + Xg+idui-iVg+ifj = (Dug-i). 



i=2,...,t 



The matrix 



/ 1 -Xg+2 ■■■ -^2g 



M:-- 



A-2 ... X g \ 
d\%2 

dl:g~\Xg 



1 



d\kg+2 1 



di :g -iX 2 g 



1 I 



is in fpjj according to Lemma 3.5. The entries of the vector v' := vM satisfy the relation 
gcd(v'j,v^ +1 ) = D\- g _\ by definition of A.,-. Therefore, there exist tifo G Z with fiv'j + 
?2Vg +1 = Di : g_i , and the matrix 7Y that differs from the unit matrix only in the entries 



(9) 



H,l «l,g+l 



J2 



^±1 



transforms v' into a vector of the form v = v'N = (Di :g -\,*, *,0, *,..., *). 
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Part (ii): 
Necessity: 

Since the divisors are invariant under the action of f J^, we must have D,-(v) = D;(w) for 
conjugate vectors v and w which obviously implies D\ :g -\(v) = D\- g -i{w). 

Now, let M e fpjj such that w — vM and let k G {l,...,2g}. To keep the notation 
easier we only consider the case k < g, the other case g < k < 2g can be treated similarly. 
By definition we know that A : g-i divides v, and v g+ ,\ From Lemma 3.5 we obtain 

8 8 

w k = m 'k V i + E m g+i-.k v 8+i + m kk.Vk 
i=l, i=l 

= £ (£)i:i_im^D i:g _iv-) + Y,(Di:i-im' g+i!k D tg -iv g+i ) + {di :k -m + l)v k 
1=1, i=i 

= + + d\ : t_iwDfc g _ i D ^'* j + Vft modDi : g_i and since D ^ v * j € Z 
= v# modA:#-i- 
Sufficiency: 

Due to part (i) we can assume v and w to be given in the form 

v= (Di :? _i(v),V2,...,v g ,0,v g+ 2,...,V2g) and 

W=(Pi :g -i(w),W2,...,Wg,0, Wg+2 ,...,W 2 g), 

where v\ =w\. Let A := D,(v) = D,(w). Since v, = w, mod A:g-i, let n, be defined by 

Wi=niD 1:g -i+Vi for i = 2, . . . ,g,g + 2, . . . ,2g. 
The matrix M defined as 



/ 1 «2 
1 



M:= 



V 





duiHg+2 

d\-g-\n2g 



n 2g \ 



1 

-<il:l«2 



1 



f= r lev 

fc 1 pol 



-d\;g-\Hg 1 / 

transforms v into vM = (Di :g _i,W2,. . . ,w g , v g +i,w g + 2 , ■ ■ ■ ,w 2g ) where 

V g+ l = duin g +2V2-\ \-dl: g -in2 g V g + 0—dl : in 2 V g+ 2 d\;g-\UgV2g- 

Since A' : g-i divides v; and Vg+i by definition, we know that At:g-i divides every term of 
v g+ \ and thus gcd(Di : g_i,Vg+i) = A:g-i- This implies that we can find a matrix N as in 
(9) which transforms vM into w and thus v and w are conjugate under f jjjj. □ 

Corollary 4.2: Set of representatives. 

A set of representatives for the orbits of f ^ is given by the vectors 

V = (A:#-l, A: A >-ia2, A:£-lfl3, ■ . . ,ag,0, A2:g-l«£+2, A3:g-1<VH3, . . . ,a 2g ) 

where {A} runs through the set of all possible divisors as given in Theorem 2.9 and 

<a, <Di : ,_i, < a g+i < Di :i -i fori = 2,...,g. 

Proof. 

This follows easily from the above Lemma 4.1 considering that by definition A: A >-1 l v i|g+i> 
and using Theorem 2.9 for the restrictions on {A}- □ 
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4.2 Orbits of isotropic lines under f po i 



Definition 4.3: Representative vectors for f p „i. 

For v = (vi, . . . , V2 g ) G I? 8 and i = 1, . . . ,g, let 

v/ := gcd(v %+1 , . . . ,v ilg+h diV i+l \ g+i+u . . . A-.g-W g \i g ) and 

v:=(v 1 ,..,v J ,0,..,0)eZ 2 *. 

In this form, adjacent entries are related in the following ways: 
Lemma 4.4: Properties of v,. 

For all primitive v G Z 2i ', the v; satisfy the following relations: 

(i) . vi = Di : g_i(v) and v g = 1 

(ii) . Vi = l,...,g-1 : Vi\d(V i+ \ 

(iii) . Vi = 2,...,g : Vi|v/_i 

(iv) . Vi = 2,...,g-1 : v / = gcd(v / _i,Vi| g+ /,d/V/+i). 

Proof. 
Part (i): 

By definition we already know that i |vi - The definition of v x immediately gives 
( D v ' - ) = / with / defined as in equation (8) on page 7. We have already proved that 

/ = (1) and hence we have vi = Di ;g -i as claimed. 

Since v is primitive, we have v g = gcd(vi , . . . , V2 g ) = 1 • 
Part (ii) and (iii): 

These follow immediately from comparing the elements of the greatest common divisors 
in the definitions of v,- and v,-+i or v,_i, respectively. 
Part (iv): 

Define v\ := gcd(vi_i,v,-^ +( -,<i ( v (+ i). From the definition of v,- and parts (iii) and (ii) we see 
that v, divides gcd(vj_i)V,-|g +l -,diVi+i) = v\. On the other hand, from the definition of v- we 
see that v\ divides gcd(v X | g+1) . . . ^^g+^^v^+i, diV i+l \ g+i+u . . .,di:g-\v g \ig) = v,-. Since 
both are positive integers, this proves equality. □ 

We now show that there is a unique v in each orbit. 

Lemma 4.5: Orbits of isotropic lines under f po i. 

Let ~ denote congruence with respect to the action of f po i. Then 

(i) . v ~ v. 

(ii) . v ~ w ^==^ v = w (here we have equality, not only congruence) 

Proof. 
Part (i): 

We prove congruence by giving matrices that transform v into v iteratively. In the ith 
step the /th component of the vector will become v, whereas the (g + ;')th component will 
become zero. The existence of such matrices is shown by induction. 

For the first step we refer to Lemma 4.1 where it has already been done using a matrix 
M G fpjj C fpoi. For the other steps we shall now construct matrices in a similar way. 
Assume that we have completed the first ;' — 1 steps and hence have a vector of the form 

V = (Vl, . . . ,Vi-l,Vf, . . . ,v g ,0,. . .,0,v g+i , . . .,v 2g ). 
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Lemma 6.2 tells us that we can find A,,- such that 



gcd(v g +i,Vi+ £ Mi:;-i v i+ E K+A-i-^g+j) 

j I 8 7=1, -,8 

= g cd ( v i|«+i ; • • • ^i\g +i ,diV i+ i\ g+i+ i, . . . ,dt g -iv g \2g) = vi- 

Since v g +\ = ■■■ = v g +i-\ = we may obviously choose X g +\ = ■■■ = X g +i-\ = 0. Now 
we can define the matrix 



M:= 



1 


h 








\ 




1 h-1 












1 













dih+i 1 













dug-iXg 


1 
















1 















1 




. 


• kg+i+i . 




* 


* 1 * 


* 










1 






di-.g-\%lg 








1 / 



Where the * in the upper right quadrant are X\ 



rfl:i -' Vs+ ' ,. . . Ai-i^ 1 ^ 1 and those in the 



V! ' > > ' Vj_l 

lower right quadrant are —d\i-\k\,. . . , — d,_iA.,_i, 1, — A.,-+i, . . . , — X g so thatM G f po i. De- 
fine w := vM. Then 

Vj for 1 < j < i 

vj ± X k v g +i for i<j<g 

forg<;'<g + ; 

Vg+i f or j=8 + i 
k vj ± 5l ft v g+ i for g + i < j < 2g 

for the appropriate indices k. Furthermore, the definition of Xj guarantees that we have 
gcd(w;,w g +,') = Vf. This shows that w = v. 

We complete the induction step using a matrix /V as in (9). 
Part (ii): 

Using (i), we immediately obtain v ~ v = w ~ w. 
=>: Since we know from part (i) that v is conjugate to v, we may assume v and w to be 
of the form v and w, respectively. Since v ~ w there exists a matrix M £ f po i such that 
w = vM. We will show that v ; divides Wj for all y = 1, . . . ,g. 
Fix j £ {I,... ,g}. We have 

vj = gcd(vi, . . .,Vj,djV j+ u. . .,d J:g -iVg) and 
= gcd(wi,. . .."-/.'//n-, ;.. . . ,<fyj-iw,) 

- 8 8 A' 8 . 

= gcd ( £ m/,iv/, . . . , £ nnjVi,dj £ m,j+iv ( -, . . . £ m/ ig v/J . 

i=i i=i ;=i i=i 

Consider a single entry in this gcd and denote it by 



Lf = i m <* v ' 



for 1 < k < j 



dj:k-lLi=l m 'k v i for j < k < 8 
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Lemma 3.3 tells us that m& — dt.i-im' ik if k < i and hence we can rewrite this as follows: 
For 1 <k< j we have 

k j g 

W k = Y, m ik v i+ E ( d k:i-im' ik )Vi+ £ {d k .j-\d j:i -im' ik )vi. 
i=\ i=k+\ !=./+! 

The summands in the first two sums each contain the factor v, with ;' < j; the summands 
of the last sum the factors djj-iVi with ; > j. A similar reasoning holds for j < k < g. We 
therefore obtain that each Wk is a multiple of gcd(vi, . . . ,v j,dj-jVj+i, . . . ,dj :g -iv g ) = Vj and 
therefore Vj\wj. 

On the other hand, since M^ 1 e f po ] and v = wM~ l we now also know that wj divides 
Vj for all j — I,... ,g, thus v = w. □ 

Corollary 4.6: Set of representatives. 

A set of representatives for the orbits of f po i is given by the vectors 

v= (Di : g_i,D 2 : g -ia2,O 3:g _ia3,...,D g _ia g _i,l,0,...,0) G I? 8 

where {£),} runs through the set of possible divisors as given in Theorem 2.9 and a, > 
with 

(10) a ! |gcd(D ( _i«(-i,^«i+i) fori = 2,...,^-l 

where we let a\ = a g = 1. 
Proof. 

The vectors v defined in Definition 4.3 can indeed be given in the form stated above: The 
factors Di-g-i must be present because of the divisibility conditions implied by the defini- 
tion. Define a, := D *' ■ . The values for a\ and a g follow from Lemma 4.4 part (i). Then 
Lemma 4.4 part (iv) shows that for i = 2, . . . ,g — 1 the condition on a, is required. 

The fact that this is indeed a set of representatives follows from the just established 
Lemma 4.5. □ 



Corollary 4.7: Coprime polarisation types. 

If the polarisation type is coprime then a, = 1 for all i = 1, . . . ,g. In particular, the orbits 
can be represented by the vectors 

v=(Di : s_i,0,...,0,l,0,...,0) 

where all D, dividing di occur without further restriction. 
Proof. 

By induction over i one can use (10) and the coprimality of the polarisation type to prove 
that for i = 2, . . . ,g— 1 we have a,|gcd(Di :i _i,a, + i). Now we can use the fact that a g = 1 
which implies recursively that indeed a, = 1 for all i = g — 1,. . . ,2. The claim follows 
from the fact that, according to Corollary 2.10, the value Di :g -\(v) determines all Z),(v) 
uniquely. 

It is obvious that Theorem 2.9 does not imply any restrictions on the D, in the coprime 
case. □ 



5 Orbits of isotropic g-spaces under f po i 

In this section we only consider types of polarisations that are square-free and coprime. For 
these polarisation types we prove that r po i acts transitively on the ^-dimensional isotropic 
subspaces of Q 2g . 
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In order to do this we consider primitive integer vectors v , . . . , v g that generate an 
isotropic subspace h = v 1 A • • • A v s C Q 2g - We may restrict the discussion to those sets 
of vectors that form a Z-basis of h% :— hC\I? s , in other words h% = 0Zv'. In this case 
primitivity with respect to hz implies primitivity with respect to I? g . 

The main point of the proof is that any h% of rank g has a basis satisfying the following 
property: 

(11) D l .. g - l {v i )=di; i - i for all *•=!,...,£. 
To construct such a basis we use two basic transformations: 

• The operation of y G r po i on all of the v l . Let v l :— y(v') for all i = 1, . . . ,g. Since 
the Di are invariant under the operation of f po i, we can find a basis of h satisfying 
property (1 1) if and only if we can find such a basis of h. 

• A linear combination of basis vectors of h given as multiplication by a unimodu- 
lar matrix A. Since A -1 exists and is an integer matrix, the vectors v, are linear 
combinations of the v, := v;A and hence the lattice h% remains unchanged by this 
transformation. Additionally, Lemma 6.3 gives the following property: assume that 
the basis transformation only involves the vectors i\,...,i n . Then 

gcd (D k (v h ), . . . ,D k (^")) = gcd (D k (v h ), . . . ,D k (J")) 

for any 1 < k < g — 1 . 

During the proofs, we shall denote the vectors after any transformation by v, but then, by 
abuse of notation, relabel them as v,-. 

In the case g = 2, this problem was treated by Friedland and Sankaran in [FS]. The 
following lemmata are generalizations of the corresponding steps to arbitrary genus. 

Lemma 5.1. 

Fix a square-free, coprime polarisation. Let h C Q 2 * be an isotropic subspace and v 1 , . . . , v g 
a Z-basis of h%. Let 2<n<g and 1 <h,...,i„ < g a set of n distinct indices. Then 

gcd {D k {v h ),..., £> t (v ! ")) = 1 foralU>£-«+l. 

Proof. 

Since the order of the vectors is irrelevant for the gcd, we may assume ij = j for all j = 
1 , . . . , n. The claim of the lemma is obviously implied by the statement 

(12) m k ~gcd(D k (v l ),...,D k (v n )) =1 fovk^g-n + l 

since higher values for k mean smaller values for n and hence we have that a set of fewer 
D k is already coprime. 

Now, the basic idea of the proof is to show that we can construct a basis vector w with 
the property that m k divides every entry. Since basis vectors are primitive, this implies that 
m k = 1 as claimed. 

We shall write the basis vectors as row vectors of a matrix, where * is to stand for any 
value in Z, » k e m k Z and x e Z\m k Z. 
Part I: 

We first bring the basis into a standard form which is given by the following description. 
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Claim 1: Let g e N and n = 2, . . . ,g. Let q :~ L 2 ^] anc ^ J = !>•••>#• Then we can 
transform the basis v 1 , . . . , v" into the following form: 

for 1 < i < j- 1: v ! ' = (*, . , *,Q, . 0, 1,0, . ,0; *, . , *, » k , . , » k ,Q, . ,0) 

g-j j-i >'-l g-y ./-' «' 

for i = v' = (*,(X_^0, 1 ,0^^,^^^ 

g-j -i j-i s 
for j+ 1 < i < n - j: V = (^^^O^^A^^,^^) 

g-j j g-j -i j 
forn-7 + 1 < i < n- 1: V = (*, . , *, 0, . , 0; 0, *, . , *, Q, . , 0, , , »^ ) 

g-y 7 g-j-l «-i y-n+i 

for ; = n: v" = (*, , *,0, ,0; *, , *, » t , . , »& ). 

g-7 g-j ) 

We fix g and prove claim 1 by considering the values n = 2,...,g separately, using 
induction over j. For j = 1, the first and fourth condition are empty and the second one is 
implied by Corollary 4.7. We transform the basis such that v 1 has the given form. To fulfil 
conditions three (if n > 3) and five we proceed as follows: 

For z = 2, ... ,7j replace v' by v' := v ! — v^v 1 such that = 0. Since v 1 A • • • A v" is an 
isotropic space, we know that for i = 2, . . . ,n 

(13) = <vV> =Di : ,-i(v 1 )v; +1 + dus-As => = -^^A- 

If all = we already have a basis satisfying conditions three and five. Otherwise we 

may assume that v\ ^ 0. For allz' = 2,...,n — 1 where v' 2 f Owe fulfil condition three 

iteratively the following way: there exist integers X,/j such that Xv' 2g +M V % = g cc '( v 2g> v %)- 
By replacing 



gcd(v^,v^ g ) gcd(v^,v^) 



and v":=^v'+/iv n 



we obtain a new basis where v' 2g — and due to (13) also v' g+l = 0. Hence, we have 
achieved that v' satisfies condition three. Note that v 2g — gcd(v' 2g ,v 2g ) ^ and so we may 
proceed with the next For condition five we use the isotropy 

(14) 0=(v\v n )=D l:g . 1 (v l )v n g+l +d 1:g - 1 v^ g =d 1:g -iv n 2g modK) 2 . 

From the facts that gcd(J r ,Wfc) = 1 for r ^ k and gcd(^,mt) = 1 since the polarisation type 
is coprime and square-free, we obtain have m^v^. This completes the proof of condition 
five for j = 1 . 

Now we continue the induction over j by assuming that claim 1 is true for some 
j = 1 , . . . , q — 1 and establish it for j + 1 . This is done by essentially the same methods 
we have used for j = 1 . Here, we use Corollary 4.7 for genus g — j to find a matrix that 
transforms v ;+1 as desired but leaves the entries g — j + 1 , . . . , g, 2g — j + 1 , . . . , 2g of all 
vectors unchanged. 
Part II: 

We are now in a position to try and transform the basis such that we obtain a basis vector 
w having the property that divides every entry of w. Recall that this proves the lemma 
since basis vectors are primitive and hence must be equal to 1 . 

Because of the entry 1 in the vectors v 1 , . . . , v q where all other vectors have zeroes, it 
does not make sense to use them in the construction of w. The other vectors are such that 
nit divides all but the critical entries v' r where k < r < g — q+1 or g + k<r< 2g — q+1 
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either by definition of nik or by construction of v'. These are exactly 28„ entries in each of 
the 8„ + 1 vectors v q+1 , . . . , v", where 8„ := |_§J — 1. 

For g < 3 we have n < 3 which gives 8„ = 0. Hence, for this case the proof is complete. 
To treat higher g we give an explicit construction for w. The methods used are basically 
the ones described before in Part I, but in order to make the proof more accessible we have 
developed the following short hand notation: First of all, notice that neither the number 
of critical entries nor that of useful vectors depends on g but only on S„; we can therefore 
work independently of g. We use the following methods to transform the basis or to gain 
information: 



M 



We use a matrix N G r po i as in (9) changing the entries in the xth column of both 
halves of each vector. This is done in such a way that v\ = and vf +x = gcd(v?, vf +x ) . 



We replace the vectors i and j by a linear combination - this is done by multi- 
plication with a unimodular matrix. After the transformation we have v\ = and 
v x - = gcd(vf,Vy). It is important to note that if v\ or v x is x, then so is v x . 



(0=1 



We use the fact that the ;'th basis vector is primitive to gain x at some entry. 
We use the isotropy (v,-, Vj) = as in (14) to gain at some entry. 



~^ We use Lemma 6.4 on v,-. This transformation involves all columns x where v x is not 
divisible by m\. Since the vector remains primitive, the gcd thus constructed can be 
written as x . (All its multiples can only be given as *.) 

For some steps to be possible we need certain entries of the basis vectors to be non-zero. 
We assume this to be the case where needed. If these entries would vanish, we could either 
alter the order of the basis vectors, skip the step in question of even arrive directly at a 
contradiction proving our claim. 

The transformations in the following construction are given in full generality. To 
illustrate the procedure, we complement it with the matrices for the case 8„ = 5. 



\ 



(2,g+l;l)(3,g+l;l) 



(2,3«- 



* 



(i,2;s+i) 



/ 00000 

* * * * 
* * * * 
* * * * 
* * * * 
* * * * 



<^2><U> 



/ 


0000 * 




\ 


* 






000* * 


if; 






* * * 






V 


* * * * 




/ 


^ ^ ^: ^: 





/ 


0000* 


* * * * 






/ 0000 


* * * * \ 




0000* 






0000* 






000* * 
* * * 






\hg\ 


000* * 
* * * 




V 


()**** 








* * * * 








J 




^ ***** 


***** J 



* * * : 

* * * * 

* * * * 

* * * * 

* * * * 



(2, g +l;2)(3,g+l;2) 



(2,4»-l)(3,4»-l) 






0* * * 


X 


* •t 


* * * * 


* 


* 9 k 


* * * * 


* 


* * * 


* * * * 


* 


* * * * 


* * * * 






* * * * 


* 



(2,3;2g) 





* m k 

* » t 

* * * 

* * * * 





* * 
*000* 

* * * 

* * * * 

* * * x 
* * * * 



* * * x 
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|2;i| 




•* 

* m k 

* * * 

* * * * 



* * * x 
* * * * 



Now repeat the following steps for i = 2, . . . ,g — 1: 




m k 



* * * 

* * * X 

* * * * 

* * * * 

* * * * 

* * * * 



<i,i+l> 



(i+l,l;2g) 



(i+l,i+2;g- 





000* * 
* * * 
* * * * 



(U+1) 



(i+l,i;2 g - 




m k 
.* 

* * * 
* * * * 

000- 



00* 
* * 

* * * 



* * * * 

* * * X 

* * * * 

* * * * 

* * * * 

* * * * 



* * * * X 

* * * x 

* * * 



* * * 

* * * 

* * * 

* * * 

* * * 

* * * 



After the repetition we obtain a matrix of the following form: 



o •* •* »k *k 
•* »k *k *k 
•* »k *k »k 
•* •* *k »k 
»k *k *k »k 



* * * * X 

* * * x 

* * x 

* x 

* 



»k »k »k *k 
•* *k *k *k 

o •* n •* •* 

o »k *k *k »k 
o •* •* »k *k 



* * * * x 

* * * x 

* * x 

* x 
x 



<g,£+i>te-U'+i> 



te+M;2g)(g+l,2|2 g -l) 



(i-s+i) 



'* •* •* •* 
'* •* •* •* 
'* •* •* •* 

•k *k *k 'k 
•k *k »k 'k 
•k *k 'k *k 



* * * * X 

* * * X 

* * x 

* x 
x 



(g+l,g;g+l) 



V 



•k 'k *k *k *k 
*k *k *k 'k *k 
*k *k *k *k *k 
*k *k *k *k *k 
*k *k 'k *k *k 
*k *k *k *k *k 



* * * * X 

* * * X 

* * x 

* x 
x 




Now, all entries of the last row vector are divisible by while it is supposed to be a 
primitive vector, giving the contradiction. □ 



Lemma 5.2. 

Fix a square-free, coprime polarisation. In any rank-n-sublattice hz C hz with 2 < n < g 
we find a vector v satisfying D g - n+ \(v) = 1. 

Proof. 

Let k := g — n + 1 and denote a basis of hz by u 1 , ... ,u n . Let m := mm{D k {u) \u G hz}- 
Now, let m 1 G hz be a primitive vector with D k {u l ) —m. We can obviously always find such 
a vector. Our aim is to show that m= 1. Since u l is primitive, [OR, Kapitel 3, Satz 10] tells 
us that we can find m 2 , . . . ,u n such that u x is a basis of hz- According to Corollary 4.7 

we can find a transformation y such that in the basis u' := yd' of yhz the kth entry of w 1 is 
u\ = D k:g _i(u l ) =mD k+ \- g _i{u x ). Note that due to the invariance of the divisors we have 
the equality m = min{D k (u)\u G yhz}. 

We modify the basis as follows: Let i = 2, . . . ,n. If the A:th entry of u' is equal to zero, 

(l,/;t) 

we leave u' unchanged. Otherwise, we use the transformation previously denoted by 

to obtain u' k = and u\ = gcd(u\,u' k ). After repeating this procedure for i = 2, . . . ,n we 
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modify the basis one more time by letting v 1 := fi 1 and V := u' + u 1 for i > 2, so that now 
the kth entries of all vectors v 1 , . . . , v" are equal to ul. 

Since for all i = 1, . . . ,n we know that D^v') divides v\ = gcA{u\, . . . ,m£) and d[ by 
definition, we may conclude thatD^v 1 ) divides gcd{dk ) u\) = gcd^^wDit+i^-^M 1 )) = m 
which implies Dk{v l ) < m. 

On the other hand, from the definition of m we know D k (v l ) > m since v' G y/zz and m 
is minimal. Therefore, D k (v l ) = m for all i — 1, . . . ,n. This shows that, using Lemma 5.1, 
m = gcd (D k (v l ), . . . ,D k (v n )) = 1 which shows that D k (u l ) = m= \ as claimed. □ 

Theorem 5.3. 

Fix a square-free, coprime polarisation. Then f po i acts transitively on the ^-dimensional 

isotropic subspaces of Q 2 * 

Proof. 

Let e k be the fcth unit vector. We want to show that, given any g-dimensional isotropic 
subspace h C Q 2g we can find a basis u l , . . . , u g of h% such that there exists a transformation 
y G fp i satisfying yu' = e' for i = 1 , . . . ,g. The proof is by induction. 

More precisely, we want to show the following for any k G {0, . . . ,g}: 
Claim 1: We can transform the basis u l , . . . , u g of hz such that 

u' = e' for i = 1 , . . . , k and 
(15) = (^^,^^,0^^,*^^ fori = k+l,...,g. 

k g—k k g—k 

For k — this is trivially true and hence we may use this as start for the induction. 
Assume that claim 1 is true for some k G {0, . . . ,g — 2}. Denote the isotropic subspace 
generated by u k+l , . ..,u g by h. Note that we may apply Lemma 5.2 for this subspace 
without losing the property (15): of the basic transformations mentioned at the beginning 
of this section only the operation of y G f po i could cause problems since it affects all basis 
vectors simultaneously. However, we may restrict ourselves to using transformations of the 
form 



/ tk 

(16) y = 



V 



\ 



B 



D J 



G r po i 



and these leave the property (15) valid. Hence, Lemma 5.2 tells us that we may assume 
(if necessary after suitable transformations) that the basis u k+1 , ... ,u 8 of hz is such that 
Di{u l ) = 1 iori = k+\,...,g-\. 

lfk = g — 2, the vector v := u 8 ~ l already has the property that Dk+\ : g-\{v) = 1. Oth- 
erwise, we let 

v— v d (n) u n 

v ■— a k+l:g-\ u ' 

n=k+i 

where d^) b := d a :c-id c +\:b- Since gc&{df^) v . . . ,4+1:^-1) = 1 we see that v is primitive 
and hence we can find a basis v k+1 ,. . . ,v g of hz where v^ 1 = v. We want to show that 
Dk+v.g-i(v) = 1 for < k< g — 1. Again, we use induction to prove 
Claim 2: For j = k, . . . ,g — 1 we have Dk+\ : j{v) = 1 . 

Again, for j — k the claim is trivially true and we have a start for the induction. Assume 
now that claim 2 is true for some j G {k, . . . ,g — 2}. Then 

D .j+i( v ) = E cd ^+i,gcd { ^ S ^) Y s= i) and since Vl l« +1 = ' ' ' = v k\g+k = 0, 

= gcd^ /+1 ,gcd(-^-V +1 
\ \D s: j(v) J s=k+\ 
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By assumption Dk+\-.j(v) = 1, which implies D s -j(v) = 1 since s > k + 1. Hence 

= gcd (dj +u gcd(v slg+s y+l + ^ 

= gcd [ d J+ i,gcd( d k+i-g-i u "\g+sY _ ) leaving out multiples of d j+ \ 

\ K n=k+l /s-k+lj 

= g cd (dj+ugcd (4+i!i-i M s|^+ s )i=* + i) and coprimality of the d t gives 
= gcd (^+i,gcd(«J^)^ +1 ) 

and since the polarisation is coprime we have gcd(d j+\,D\-j(u^ +l )) = 1 and therefore 



= gcd ^ +1 ,gcd and since = ■■■= u^ +k = 0, 



This shows that claim 2 is true for j+l, completing the proof that -D/t+i-^— l (v) = 1 f° r an Y 

*e{o,...,*-i}. 

Hence, we can find y G f po i of the form (16) such that yv = e k +i . Under this operation 
the basis v k+l . . ,v g of is transformed into a basis of yhz which we shall, by abuse of 
notation, again denote by v k+l , . ..,v g . Note that now v k+l = e k +\. Since yh is again an 
isotropic subspace, we have for j = k + 2, . . . ,g: 

(17) 0^(v k+i y)^d 1:k -l-v J g+k+1 => v j g+k+l =0. 

Thus, we obtain a basis u k+l := v k+1 1 u l := v' — v l k+1 v k+l satisfying claim 1 for k+ 1. This 
completes the induction. 

Now that we have reached (15) for k = g — 1 it is easy to see that we only need one 
more transformation of the form (16) (where the matrices A to D are just integers) to prove 
claim 1 for k~g. 

Since we have now shown that for any ^-dimensional isotropic subspace h we can 
find a basis of h% that can be transformed into ei,...,e g by the action of an element in r po i, 
we have proved the transitivity of the group action. Note that this basis indeed satisfies 
property (11). □ 



6 Appendix: Technical lemmata 

Lemma 6.1. 

Let g,d EN and A = (ay) e Z gxg . If there exists k E {1,. . . ,g} such that for all i,j satis- 
fying 1 <j<k<i<gwe have d|a (; -, then d\ det(A). 

Proof. 

For g = 1 the claim is trivial. The induction follows easily by developing along the kth 
column, since either d\ai^ or the assumption gives d\ det(A('^). □ 

Lemma 6.2. 

Let xi, X2 and yi, . . . ,yi be integers with x\ ^ and gcd(xi,X2,yi, . . . ,y,) = d 6 N. Then 
there exist integers OCi , . . . , 0C< € Z such that gcd(xi ,X2 + OCiji H h «,->',■ ) = d. 
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Proof. 

This is a fairly straightforward generalisation of [HKW, Part I, Lemma 3.35]. □ 
Lemma 6.3. 

Assume we are given a coprime polarisation type, vectors v 1 , . . . , v" <G l? s and a unimodular 
integer matrix A. Consider the basis transformation U := AV where m, and v, are the row 
vectors of U and V, respectively. Then 

gcd (D k {u 1 ),. . . ,D k (u n )) = gcd (Dkiv 1 ),. . . ,D k (V)) 

for any 1 < k < g — 1 . 
Proof. 

Assume the notation A = {an). The y'th entry of the z'th vector is given by u l - — Yl{=\ a u vl j 
and hence gcd(v' )" =1 divides u'j for all i. Since the polarisation type is coprime we have 
gcd(dk,D r (u')) = 1 for r ^ k which implies D k (u l ) = gcd(<4,gcd(V.| g+ ^)j =1 ) and so 



gcd (D k (v% =1 -gcd (d k ,gcd (gcd (4 + ^ =1 )" = 

= gcd ^4, gcd (gcd {v)\g + j)"=i^ ._^J which divides 

gcd(*,gcd(gcd(M} k+/ .)J =1 )" =i )=gcd(D,( M s ))" = r 

Since A -1 is also a unimodular integer matrix we also obtain divisibility in the other direc- 
tion, and since both numbers are positive integers this implies equality. □ 

Lemma 6.4. 

Assume g > 2 with any polarisation type and v — (vi,. . . ,v g ,0, . . . ,0) G I? g . Then 
there exists a matrix M £ f po i such that for u — {u\,. . . ,U2 g ) ■— vM G l? s we have 
u g = gcd(vi, . . . ,v g ). Furthermore, M can be chosen such that it is an automorphism of 
the sublattices IA x {0}« C I? g and {0}*' x U c 1 2g . 

If we choose a set of indices 1 < i\ < ■ ■ ■ < i n < g then there exists M G f po ] such that 
Ui n = gcd(v, 1 , . . . , Vi n ) and M is an automorphism of the sublattices 7 -e !y Z and 7 e ?+( - .Z 
where e,-. is the ijth unit vector. 

Proof. 
Claim 1: 

Assume a,b,d G Z given. Let A = diag(l,c/). Then there exists a matrix G G §D(A) such 
that (a 7 b)G = (u,v) with v = gcd(a,b). 

We prove this as follows: denote x := gcd(a,b). Then there exist integers a, p G Z 
such that <xa + fib = x. Chose t to be the product of all primes dividing d but not dividing 
p. Then it can easily be seen that gcd(p — f|,<i(a + f|)) = 1. Hence there exist integers 

G Z with A-(p-ff) -fid(a + t^) = 1 and the matrix 

G=( X a + t ^) 

satisfies the properties claimed. 
Claim 2: 

Assume g > 2, let (l,di,...,di : g-i) be any polarisation type and A the diagonal matrix 
corresponding to it. For any v = (vi, . . . ,v g ) G IP we can find a matrix G G §D(A) such 
that u :— vG satisfies u g — gcd(vi, . . . ,v g ). 
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The proof is by induction and shows that G can be chosen to be of the form 



/ Pi 




(18) 












oti \ 

a 2 





\d\:g-\1\ d 2 : g -lj2 



d g -ij g -i a g ) 



For g = 2 this is exactly Claim 1 . For the induction, fix any g > 2 and assume we can 
find G g of the form (18) satisfying detG g = 1 and £? =] a,v, = gcd(v! ,...,v g 



and v 



(V0)V1,. 

-1 , 



Now let the 



polarisation type for g+ 1 be given by (l,Jo,^0:i, • • ■ ,^0:g-i 

We use Claim 1 with a = v$,b = gcd(vi, . . . ,v g ) and d = do :g -\ Ylf=i Pi t0 obtain a 
matrix G' = ( satisfying detG' = 1 and Xovq + X\ gcd(vi , . . . ,v g ) = gcd(vo, ...,v g ). 
Define the matrix G g+ \ to be 



G g+ i := 



m 
o 
o 



o 
Pi 








p 2 



A-ioci 
A,ia 2 





\d 0:g -m di :g -iYi 



p g _i Xia g _i 

d g -2: g -iy g -2 dg-flg-i Xia g J 



Some simple calculation shows that G g+ i is as claimed. 

Now we can conclude the proof of the lemma. Use Claim 2 to obtain a matrix 
G G SD(A) satisfying u' g = gcd(vi, . . . ,v g ) for u' := (vi, . . . ,v^)G. Since SD(A) is a multi- 
plicative group, G _1 e SD(A). Now, M = ( q G -i ) satisfies the properties claimed. 

This last step goes through the same if we restrict everything to the sublattice 
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